Tradeoffs for nearest neighbors on the sphere

We consider tradeoffs between the query and update complexities for the (approximate) nearest neighbor problem on the sphere, extending the spherical filters recently introduced by [Becker–Ducas–Gama– Laarhoven, SODA’16] to sparse regimes and generalizing the scheme and analysis to account for different tradeoffs. In a nutshell, for the sparse regime the tradeoff between the query complexity nq and update complexity nu for data sets of size n can be summarized by the following equation in terms of the approximation factor c and the exponents ρq and ρu: c √ ρq + (c 2 − 1)ρu = √ 2c2 − 1. For small c = 1 + ε, minimizing the time for updates leads to a linear space complexity at the cost of a query time complexity of approximately n1−4ε 2 . Balancing the query and update costs leads to optimal complexities of n 2−1), matching lower bounds from [Andoni–Razenshteyn, 2015] and [Dubiner, IEEE Trans. Inf. Theory 2010] and matching the asymptotic complexities previously obtained by [Andoni– Razenshteyn, STOC’15] and [Andoni–Indyk–Laarhoven–Razenshteyn–Schmidt, NIPS’15]. A subpolynomial query time complexity n can be achieved at the cost of a space complexity of the order n 2), matching the lower bound n ) of [Andoni–Indyk–Pǎtraşcu, FOCS’06] and [Panigrahy–Talwar–Wieder, FOCS’10] and improving upon results of [Indyk–Motwani, STOC’98] and [Kushilevitz–Ostrovsky–Rabani, STOC’98] with a considerably smaller leading constant in the exponent. For large c, minimizing the update complexity results in a query complexity of n 2+O(1/c4), improving upon the related asymptotic exponent for large c of [Kapralov, PODS’15] by a factor 2, and matching the lower bound n ) of [Panigrahy–Talwar–Wieder, FOCS’08]. Balancing the costs leads to optimal complexities of the order n 2−1), while a minimum query time complexity can be achieved with update and space complexities of approximately n +O(1/c) and n 2+O(1/c4), also improving upon the previous best exponents of Kapralov by a factor 2 for large n and c. For the regime where n is exponential in the dimension, we obtain further improvements compared to results obtained with locality-sensitive hashing. We provide explicit expressions for the query and update complexities in terms of the approximation factor c and the chosen tradeoff, and we derive asymptotic results for the case of the highest possible density for random data sets.